Supervised Machine Learning

Table of contents

• 1 Introduction to Machine Learning
  • 1.1 What is Machine Learning?
    • 1.1.1 Definition (Plain language):
    • 1.1.2 Definition (Technical):
  • 1.2 Types of Machine Learning
    • 1.2.1 Supervised Learning
    • 1.2.2 Unsupervised Learning
    • 1.2.3 Reinforcement Learning (Optional for intro)
  • 1.3 The Machine Learning Workflow: A Step-by-Step Guide
    • 1.3.1 Step 1: Define the Problem
    • 1.3.2 Step 2: Collect and Understand the Data
    • 1.3.3 Step 3: Preprocess the Data
    • 1.3.4 Step 4: Split the Dataset
    • 1.3.5 Step 5: Choose a Model
    • 1.3.6 Step 6: Train the Model
    • 1.3.7 Step 7: Evaluate the Model
    • 1.3.8 Step 8: Tune Hyperparameters
    • 1.3.9 Step 9: Deploy or Interpret the Model
    • 1.3.10 Step 10: Monitor and Maintain
• 2 Fundamentals of ML
  • 2.1 A Gentle Introduction | Video (12:44)
  • 2.2 Confusion Matrix | Video (7:12)
  • 2.3 Sensitivity and Specificity | Video (11:46)
  • 2.4 Expected Value | Video (13:39)
  • 2.5 Entropy | Video (16:34)
• 3 Regression Trees: Boston Housing Data
  • 3.1 Decision Trees | Video (17:21)
  • 3.2 Regression Trees | Video (22:32)
  • 3.3 What Are Regression Trees?
    • 3.3.1 Definition
    • 3.3.2 Intuition
  • 3.4 When Should You Use Regression Trees?
    • 3.4.1 Ideal Scenarios:
    • 3.4.2 When Not Ideal:
  • 3.5 How Regression Trees Work
    • 3.5.1 Splitting
    • 3.5.2 Recursion
    • 3.5.3 Prediction
    • 3.5.4 Pruning
  • 3.6 Typical Workflow for Using Regression Trees
    • 3.6.1 Step 1: Define the Problem
    • 3.6.2 Step 2: Data Preparation
    • 3.6.3 Step 3: Split the Data
    • 3.6.4 Step 4: Fit the Model
    • 3.6.5 Step 5: Predict and Evaluate
    • 3.6.6 Step 6: Visualize Tree
  • 3.7 Comparison to Other Methods
  • 3.8 Enhancements & Variants
  • 3.9 Limitations and Pitfalls
  • 3.10 Use Cases
  • 3.11 Regression Trees & Random Forest
• 4 Gradient Boosting - Boston Housing Data
  • 4.1 AdaBoost | Video (20:53)
  • 4.2 Gradient Boost Part 1/4: Regression Main Ideas | Video (15:51)
  • 4.3 Gradient Boost Part 2 (of 4): Regression Details | Video (26:45)
  • 4.4 Gradient Boost Part 3 (of 4): Classification | Video (17:02)
  • 4.5 Gradient Boost Part 4 (of 4): Classification Details | Video (36:59)
• 5 Classification Trees for Buy-No Buy Data: Eight product categories
  • 5.1 Decision and Classification Trees | Video (18:08)
• 6 Random Forest NOBuy- Buy Data
  • 6.1 Random Forests Part 1| Video (9:53)
  • 6.2 Random Forests Part 2 | Video (11:52)
• 7 Random Forest: Predictive Accuracy in IRI data
• 8 XGBoot: Conceptual Understanding
  • 8.1 XGBoost Part 1 (of 4): Regression | Video (25:45)
  • 8.2 XGBoost Part 2 (of 4): Classification | Video (25:17)
  • 8.3 XGBoost Part 3 (of 4): Mathematical Details | Video (27:23) (Optional)
  • 8.4 Quantiles and Percentiles | Video (6:29)
  • 8.5 XGBoost Part 4 (of 4): Crazy Cool Optimizations | Video (24:26)
• 9 Extreme Gradient Boosting: IRI data, Buy or NO BUy
• 10 Extreme Gradient Boosting: nhefs data

Installing packages

#install.packages("NbClust")
#install.packages("mclust")
#install.packages("factoextra")
#install.packages("DiagrammeR")
#install.packages("rfUtilities")

1 Introduction to Machine Learning

1.1 What is Machine Learning?

1.1.1 Definition (Plain language):

Machine Learning is a field of study that gives computers the ability to learn from data and make predictions or decisions without being explicitly programmed for each specific task.

1.1.2 Definition (Technical):

Machine Learning is the study of algorithms and statistical models that computer systems use to learn patterns from data and make predictions or classifications based on that learning.

1.2 Types of Machine Learning

ML algorithms are generally classified into three main categories, based on the type of feedback they receive from the data:

1.2.1 Supervised Learning

• What: Learn from labeled data (input-output pairs)

• Goal: Predict output from new inputs

• Examples:

-    Regression (predict continuous values)

-    Classification (predict categories)

Task	Algorithms
Regression	Linear regression, LASSO, Ridge, SVR, Random Forest Regressor
Classification	Logistic regression, K-NN, Decision Trees, Random Forest, SVM, Gradient Boosting, Naive Bayes

1.2.2 Unsupervised Learning

• What: Learn patterns or structure from unlabeled data

• Goal: Find hidden patterns, groups, or structures

• Examples:

-    Clustering

-    Dimensionality Reduction

Task	Algorithms
Clustering	K-Means, DBSCAN, Hierarchical Clustering
Dim. Reduction	PCA, t-SNE, UMAP

1.2.3 Reinforcement Learning (Optional for intro)

• What: Learn by interacting with an environment and receiving feedback in the form of rewards or penalties

• Goal: Learn optimal decision strategies

• Examples: Q-Learning, Deep Q-Networks (DQN)

1.3 The Machine Learning Workflow: A Step-by-Step Guide

Below is a typical ML workflow that mirrors real-world projects. It’s helpful to explain what is done and why at each stage.

1.3.1 Step 1: Define the Problem

• Is it regression? classification? clustering?

• What’s the goal of the prediction or analysis?

1.3.2 Step 2: Collect and Understand the Data

• Look at variable types, distributions, missing values

• Ask: Is the dataset balanced? What’s the sample size?

1.3.3 Step 3: Preprocess the Data

• Clean missing values, remove duplicates

• Encode categorical variables (e.g., one-hot encoding)

• Scale features (e.g., standardization)

• Create new features (feature engineering)

1.3.4 Step 4: Split the Dataset

• Training set: used to train the model

• Test set: used to evaluate final model

• Optional: use a validation set or cross-validation

1.3.5 Step 5: Choose a Model

• Match the algorithm to your problem type (classification, regression, etc.)

• Start simple (e.g., logistic or linear regression), then try more complex ones (e.g., tree-based methods)

1.3.6 Step 6: Train the Model

• Fit the model on the training data

• Use cross-validation to assess performance and tune parameters

1.3.7 Step 7: Evaluate the Model

• For classification: accuracy, precision, recall, F1-score, ROC-AUC

• For regression: RMSE, MAE, \(R^2\)

• Visualize confusion matrix, residual plots, etc.

1.3.8 Step 8: Tune Hyperparameters

• Use techniques like:

-    Grid search

-    Random search

-   Bayesian optimization

1.3.9 Step 9: Deploy or Interpret the Model

• Deploy to a production environment

• Or: explain the model using SHAP, LIME, feature importance

1.3.10 Step 10: Monitor and Maintain

• Monitor prediction quality over time

• Re-train when new data becomes available

2 Fundamentals of ML

2.1 A Gentle Introduction | Video (12:44)

2.2 Confusion Matrix | Video (7:12)

2.3 Sensitivity and Specificity | Video (11:46)

2.4 Expected Value | Video (13:39)

2.5 Entropy | Video (16:34)

3 Regression Trees: Boston Housing Data

3.1 Decision Trees | Video (17:21)

3.2 Regression Trees | Video (22:32)

3.3 What Are Regression Trees?

3.3.1 Definition

A regression tree is a supervised learning algorithm used to predict continuous outcomes. It works by recursively splitting the dataset into branches based on feature values, producing a tree-like model where each leaf gives a predicted value (typically the average outcome in that region).

3.3.2 Intuition

Imagine a game of “20 Questions” where each question narrows down the value you’re trying to predict. A regression tree asks yes/no (or inequality) questions about input features (e.g., “Is income > $50,000?”). Each answer leads to a different branch, and at the end (leaf node), you predict the mean of observed outcomes in that group.

3.4 When Should You Use Regression Trees?

3.4.1 Ideal Scenarios:

• Your outcome is continuous.

• You expect nonlinear relationships between predictors and the outcome.

• You want interpretable results with “if-then” logic.

• You have a mix of categorical and numerical features.

• You want a model that requires little preprocessing (no scaling, encoding).

3.4.2 When Not Ideal:

• You need the highest predictive accuracy.

• Your dataset is very small and prone to overfitting.

• You need smooth, stable predictions (trees can be unstable).

3.5 How Regression Trees Work

3.5.1 Splitting

• At each node, the algorithm selects the feature and threshold that minimizes Residual Sum of Squares (RSS):

3.5.2 Recursion

• The algorithm recursively splits the data, building branches until a stopping rule is met (e.g., min node size, max depth).

3.5.3 Prediction

• For a new observation, the tree traverses nodes based on the observation’s values and returns the mean value of the terminal (leaf) node.

3.5.4 Pruning

• Trees often overfit. To prevent this:

  • Pre-prune: Stop growing early (e.g., limit depth)

  • Post-prune: Grow full tree, then cut back using cross-validation

3.6 Typical Workflow for Using Regression Trees

3.6.1 Step 1: Define the Problem

• Is it a regression problem (continuous outcome)?

3.6.2 Step 2: Data Preparation

• Handle missing values (trees can often handle them natively)

• No need for scaling or dummy coding

3.6.3 Step 3: Split the Data

• Training and test sets (and optionally validation set or cross-validation)

3.6.4 Step 4: Fit the Model

library(rpart) 
model <- rpart(outcome ~ predictors, data = train_data, method = "anova")

3.6.5 Step 5: Predict and Evaluate

preds <- predict(model, newdata = test_data) rmse <- sqrt(mean((test_data$y - preds)^2))

3.6.6 Step 6: Visualize Tree

rpart.plot::rpart.plot(model)

3.7 Comparison to Other Methods

Feature	Regression Tree	Linear Regression	Random Forest / Boosting
Captures nonlinearity	✅ Yes	❌ No (unless transformed)	✅ Yes
Handles feature interaction	✅ Implicitly	❌ No (needs manual terms)	✅ Very well
Needs feature scaling	❌ No	✅ Yes	❌ No
Easy to interpret	✅ Yes (rules)	✅ Yes (coefficients)	❌ No
Accuracy	⚪ Moderate	⚪ Moderate	✅ High
Overfitting risk	✅ High if unpruned	✅ High	❌ Lower

3.8 Enhancements & Variants

• Random Forests: Ensembles of many trees with bootstrapped samples

• Gradient Boosting: Sequential trees fix errors from previous ones

• Model Trees: Fit linear models at leaves

• Oblique Trees: Use linear combinations to split, not single features

3.9 Limitations and Pitfalls

• Overfitting: Deep trees can memorize training data

• Instability: Small changes in data can produce very different trees

• Bias in split selection: Variables with more values are favored

• Poor extrapolation: Trees can’t predict beyond seen feature ranges

3.10 Use Cases

Domain	Example Task
Real estate	Predict housing prices from features
Education	Predict student performance based on inputs
Marketing	Forecast customer spending
Healthcare	Estimate patient risk scores

3.11 Regression Trees & Random Forest

data("Boston")

set.seed(123)
# default RF model: ntree = 500, mtry = nos. of features/3
m1 <- randomForest(formula = log(medv) ~ .,  data = Boston, importance=TRUE)
plot(m1)

which.min(m1$mse)

[1] 500

# Tuning mtry
ForestTune <- caret::train(y=log(Boston[,14]),
                  x=Boston[,1:13],
                  tuneGrid=data.frame(mtry=1:10),
                  method="rf",
                  ntree=500,
                  trControl=trainControl(method="oob"))
ForestTune

Random Forest 

506 samples
 13 predictor

No pre-processing
Resampling results across tuning parameters:

  mtry  RMSE       Rsquared 
   1    0.1970465  0.7671552
   2    0.1592069  0.8479968
   3    0.1492000  0.8665045
   4    0.1448385  0.8741952
   5    0.1435082  0.8764956
   6    0.1442932  0.8751408
   7    0.1456864  0.8727179
   8    0.1450441  0.8738378
   9    0.1482671  0.8681688
  10    0.1481130  0.8684425

RMSE was used to select the optimal model using the smallest value.
The final value used for the model was mtry = 5.

plot(ForestTune)

m1 <- randomForest(formula = log(medv) ~ ., data = Boston, 
                   ntree=500, mtry = 5, importance=TRUE)
importance(m1)

          %IncMSE IncNodePurity
crim    22.045081    10.2428053
zn       5.261208     0.2064273
indus   10.185472     2.2999356
chas     3.123448     0.2268523
nox     26.029882     8.4265352
rm      41.678157    16.5194111
age     17.562822     1.9765128
dis     21.341995     4.5182532
rad      7.947701     0.6057602
tax     13.035036     2.5129642
ptratio 14.317925     4.3062644
black   15.466762     1.6596508
lstat   38.190017    29.1222391

varImpPlot(m1)

plot(m1)

which.min(m1$mse)

[1] 492

# Partial dependance plots
par(mfrow=c(1,1))

# Partial plots of single variable
m1 %>%  
  partial(pred.var = "rm") %>%
  plotPartial(smooth = FALSE, lwd=3, 
              ylab = "Median log Price",
              main = "Partial Dependance Plot", 
              xlab ="Number of Rooms")

partial(m1, pred.var = "rm", plot = TRUE, rug = TRUE) #rug: 

partialPlot(m1, pred.data = Boston, x.var = "rm", rug = TRUE)

# Partial plots of two variables
pd <- partial(m1, pred.var = c("lstat", "rm"))
plotPartial(pd)

# Add contour lines and use a different color palette
rwb <- colorRampPalette(c("red", "white", "blue")) # red=low price; blue=high price
plotPartial(pd, contour = TRUE, col.regions = rwb, 
            main ="PDP: % lower status pop vs nos. room", 
            xlab ="% lower status of Pop" ,
            ylab="number of rooms")

4 Gradient Boosting - Boston Housing Data

4.1 AdaBoost | Video (20:53)

4.2 Gradient Boost Part 1/4: Regression Main Ideas | Video (15:51)

4.3 Gradient Boost Part 2 (of 4): Regression Details | Video (26:45)

4.4 Gradient Boost Part 3 (of 4): Classification | Video (17:02)

4.5 Gradient Boost Part 4 (of 4): Classification Details | Video (36:59)

set.seed(123)

# train GBM model
gbm1 <- gbm(
  formula = log(medv) ~ .,
  distribution = "gaussian",
  data = Boston,
  n.trees = 2000,
  interaction.depth = 1,
  shrinkage = 0.001,
  cv.folds = 5,
  n.cores = NULL, # will use all cores by default
  verbose = FALSE
  ) 

# Using caret with the default grid to optimize tune parameters automatically
# GBM Tuning parameters:
# n.trees (# Boosting Iterations)
# interaction.depth (Max Tree Depth)
# shrinkage (Shrinkage)
# n.minobsinnode (Min. Terminal Node Size)


# Tuning hyper hyper parameters
hyper_grid <- expand.grid(
  n.trees = c(1000, 2000),
  shrinkage = c(0.001, .01, .05),
  interaction.depth = c(1, 2, 3),
  n.minobsinnode = c(10, 20))
  
trainControl <- trainControl(method="cv", number=10)

set.seed(99)
gbm.caret <- train(log(medv) ~ .
                   , data=Boston
                   , distribution="gaussian"
                   , method="gbm"
                   , trControl=trainControl
                   , verbose=FALSE
                   , tuneGrid=hyper_grid
                   )                  

# attributes(gbm.caret)
gbm.caret$bestTune

   n.trees interaction.depth shrinkage n.minobsinnode
30    2000                 2      0.05             10

# GBM with tunes hyper parameters
gbm2 <- gbm(
  formula = log(medv) ~ .,
  distribution = "gaussian",
  data = Boston,
  n.trees = 2000,
  interaction.depth = 2,
  shrinkage = 0.05,
  n.minobsinnode = 10,
  cv.folds = 5,
  n.cores = NULL, # will use all cores by default
  verbose = FALSE
  ) 
# Variable importance
vip::vip(gbm2)

# PDPs
gbm2 %>%
  partial(pred.var = "rm", n.trees = gbm2$n.trees, grid.resolution = 100) %>%
  autoplot(rug = TRUE, train = Boston)

gbm2 %>%
  partial(pred.var = c("rm", "lstat"), n.trees = gbm2$n.trees, grid.resolution = 100) %>%
  autoplot(rug = TRUE, train = Boston)

pd <- partial(gbm2, pred.var = c("rm","lstat"), n.trees = gbm2$n.trees, grid.resolution = 100) 
# Add contour lines and use a different color palette
rwb <- colorRampPalette(c("red", "white", "blue"))
plotPartial(pd, contour = TRUE, col.regions = rwb, 
            main ="PDP: % lower status pop vs nos. room", xlab="% lower status of Pop",  ylab ="number of rooms")

5 Classification Trees for Buy-No Buy Data: Eight product categories

5.1 Decision and Classification Trees | Video (18:08)

data1 = read.csv(here("data", "iridata.csv"))

# Baby,Coffee,Detergent,Paste,Sauce,Snacks,Sunscreen,Tuna
data2 = subset(data1, subset=catno =="Baby")
str(data2)

'data.frame':   1225 obs. of  12 variables:
 $ catno        : chr  "Baby" "Baby" "Baby" "Baby" ...
 $ hhinc        : chr  "$100 plus" "$25-49K" "$100 plus" "< 25K" ...
 $ hhsize       : chr  "1-2  members" "3  members" "1-2  members" "1-2  members" ...
 $ race         : chr  "Other" "Caucasian" "Caucasian" "Caucasian" ...
 $ ethnicity    : chr  "Non-Hispanic" "Non-Hispanic" "Non-Hispanic" "Non-Hispanic" ...
 $ affluence    : chr  "Getting By" "Living Comfortably" "Getting By" "Doing Well" ...
 $ pres_child   : chr  "No" "No" "No" "No" ...
 $ age_head     : chr  "55-64Y" "50-54Y" "65+Y" "65+Y" ...
 $ educ_fem_head: chr  "Some College" "Some or High School" "Post College Grade" "Some or High School" ...
 $ census       : chr  "West" "South" "West" "Central" ...
 $ buy          : int  0 0 0 0 0 0 0 0 0 0 ...
 $ purchase     : chr  "NO" "NO" "NO" "NO" ...

cat_vars <- c("hhinc", "hhsize", "race", "ethnicity", "affluence", "pres_child", 
              "age_head", "educ_fem_head", "census", "purchase")

data <-  data2 %>%
  mutate(across(all_of(cat_vars), as_factor))
str(data)

'data.frame':   1225 obs. of  12 variables:
 $ catno        : chr  "Baby" "Baby" "Baby" "Baby" ...
 $ hhinc        : Factor w/ 5 levels "$100 plus","$25-49K",..: 1 2 1 3 3 4 2 1 4 2 ...
 $ hhsize       : Factor w/ 3 levels "1-2  members",..: 1 2 1 1 1 3 1 3 1 3 ...
 $ race         : Factor w/ 2 levels "Other","Caucasian": 1 2 2 2 2 2 1 2 2 2 ...
 $ ethnicity    : Factor w/ 2 levels "Non-Hispanic",..: 1 1 1 1 1 1 1 1 1 1 ...
 $ affluence    : Factor w/ 3 levels "Getting By","Living Comfortably",..: 1 2 1 3 3 1 1 1 1 3 ...
 $ pres_child   : Factor w/ 2 levels "No","Yes": 1 1 1 1 1 2 1 2 1 2 ...
 $ age_head     : Factor w/ 6 levels "55-64Y","50-54Y",..: 1 2 3 3 2 4 1 1 1 5 ...
 $ educ_fem_head: Factor w/ 4 levels "Some College",..: 1 2 3 2 2 2 1 3 2 2 ...
 $ census       : Factor w/ 4 levels "West","South",..: 1 2 1 3 1 1 2 3 2 3 ...
 $ buy          : int  0 0 0 0 0 0 0 0 0 0 ...
 $ purchase     : Factor w/ 2 levels "NO","YES": 1 1 1 1 1 1 1 1 1 1 ...

# Single Tree: Tuning complexity parameter, cp
TreeTune<-train(y=data[,12],
                x=data[,2:10],
                tuneGrid=data.frame(cp=seq(.001,.05,length.out = 20)),
                method="rpart",
                trControl=trainControl(method="repeatedcv",repeats=10,number=10))
TreeTune$bestTune

          cp
7 0.01647368

plot(TreeTune)

# Tree
GoodTree <- rpart(purchase ~ hhinc + hhsize + race + ethnicity +
                affluence + pres_child + age_head + educ_fem_head +
                census,data=data,method="class",
                control=rpart.control(cp=0.01389474))

# summary(GoodTree) 

fancyRpartPlot(GoodTree)

onetree <- predict(GoodTree, type="prob")[,2]

rpart.plot(GoodTree, type =5, extra=104, fallen.leaves=T, main="Sample Single Tree")

6 Random Forest NOBuy- Buy Data

6.1 Random Forests Part 1| Video (9:53)

6.2 Random Forests Part 2 | Video (11:52)

# Tuning mtry
ForestTune <- caret::train(y=data[,12],
                  x=data[,2:10],
                  tuneGrid=data.frame(mtry=1:6),
                  method="rf",ntree=500,
                  trControl=trainControl(method="oob"))
ForestTune$results

   Accuracy     Kappa mtry
1 0.6653061 0.1179931    1
2 0.6693878 0.2433716    2
3 0.6465306 0.1980087    3
4 0.6391837 0.1904641    4
5 0.6359184 0.1865918    5
6 0.6293878 0.1789447    6

plot(ForestTune)

m2 <- randomForest(x=data[,2:10],y=data[,12],data=data, 
                        type="classification",ntree=500,mtry=2, importance = TRUE)

# Random Forest prediction
rfpred <- predict(m2,type="prob")[,2]
importance(m2)

                      NO        YES MeanDecreaseAccuracy MeanDecreaseGini
hhinc          9.9518220 -0.3497756            7.2112029        35.142585
hhsize        10.9960555 10.5337820           18.7930710        29.110216
race           1.4224556  0.8806341            1.5690573        12.071906
ethnicity      0.1589872 -3.6983692           -3.0587445         8.718574
affluence      3.7526294 -0.8801165            2.3765147        17.028193
pres_child    10.4635617 13.8523152           24.9207270        25.745736
age_head      25.9393002 11.2475624           29.6558953        70.898330
educ_fem_head -5.1915254  4.2517352           -0.1875437        31.587706
census        -1.9234697  1.0663672           -0.3993511        32.299178

varImpPlot(m2)

# PDP for single variable
partialPlot(x=m2, pred.data=data, x.var=hhsize, which.class="YES")

partialPlot(x=m2, pred.data=data, x.var=pres_child, which.class="YES")

# Household Size
m2 %>% 
  partial(pred.var="hhsize", prob=TRUE, which.class ="YES" ) %>%
  plotPartial(smooth=FALSE, lwd=2, ylab=expression(probability),ylim=c(0.5,0.9), main=expression(Partial.Dependance.Plot(Random.Forest)), xlab=expression(Household.Size))

# Age of Head
m2 %>% 
  partial(pred.var="pres_child", prob=TRUE, which.class ="YES" ) %>%
  plotPartial(smooth=TRUE, lwd=2, ylab=expression(probability),  ylim=c(0.1,1), main=expression(Partial.Dependance.Plot(Random.Forest)), xlab=expression(Age.Head))

# PDP for two variables
m2 %>% 
  partial(pred.var=c("pres_child", "hhinc"), prob=TRUE, which.class ="YES") %>%
  plotPartial(smooth=TRUE, lwd=2, ylab=expression(probability), ylim=c(0.1,1), main=expression(Partial.Dependance.Plot(Random.Forest)), xlab=expression(Presence.Child))

7 Random Forest: Predictive Accuracy in IRI data

set.seed(1001)
 indxTrain <- createDataPartition(y = data$purchase,p = 0.70,list = FALSE)
 training <- data[indxTrain,]
 testing <- data[-indxTrain,] 

rfarrest <- randomForest(x=training[,2:10],y=training[,12],data=training, 
                        type="classification",ntree=1000,mtry=2, importance = TRUE)
rfarrest$confusion

     NO YES class.error
NO  126 179   0.5868852
YES  97 456   0.1754069

# Predictive accuracy in training sample
accuracy.train <- rfarrest %>% predict(training)
confusionMatrix(accuracy.train, training$purchase)

Confusion Matrix and Statistics

          Reference
Prediction  NO YES
       NO  212  32
       YES  93 521
                                          
               Accuracy : 0.8543          
                 95% CI : (0.8289, 0.8773)
    No Information Rate : 0.6445          
    P-Value [Acc > NIR] : < 2.2e-16       
                                          
                  Kappa : 0.6671          
                                          
 Mcnemar's Test P-Value : 8.025e-08       
                                          
            Sensitivity : 0.6951          
            Specificity : 0.9421          
         Pos Pred Value : 0.8689          
         Neg Pred Value : 0.8485          
             Prevalence : 0.3555          
         Detection Rate : 0.2471          
   Detection Prevalence : 0.2844          
      Balanced Accuracy : 0.8186          
                                          
       'Positive' Class : NO              
                                          

# Predictive accuracy in testing sample 
accuracy.test <- rfarrest %>% predict(testing)
confusionMatrix(accuracy.test, testing$purchase)

Confusion Matrix and Statistics

          Reference
Prediction  NO YES
       NO   50  47
       YES  80 190
                                          
               Accuracy : 0.654           
                 95% CI : (0.6028, 0.7026)
    No Information Rate : 0.6458          
    P-Value [Acc > NIR] : 0.394399        
                                          
                  Kappa : 0.1976          
                                          
 Mcnemar's Test P-Value : 0.004518        
                                          
            Sensitivity : 0.3846          
            Specificity : 0.8017          
         Pos Pred Value : 0.5155          
         Neg Pred Value : 0.7037          
             Prevalence : 0.3542          
         Detection Rate : 0.1362          
   Detection Prevalence : 0.2643          
      Balanced Accuracy : 0.5932          
                                          
       'Positive' Class : NO              
                                          

8 XGBoot: Conceptual Understanding

8.1 XGBoost Part 1 (of 4): Regression | Video (25:45)

8.2 XGBoost Part 2 (of 4): Classification | Video (25:17)

8.3 XGBoost Part 3 (of 4): Mathematical Details | Video (27:23) (Optional)

8.4 Quantiles and Percentiles | Video (6:29)

8.5 XGBoost Part 4 (of 4): Crazy Cool Optimizations | Video (24:26)

9 Extreme Gradient Boosting: IRI data, Buy or NO BUy

# create tuning grid
grid_default <- expand.grid(nrounds = c(200,300),
max_depth = c(4,5), eta = c(0.01), gamma = c(0,1), min_child_weight = c(10, 25),
colsample_bytree = c(0.7), subsample = c(0.6))

set.seed(123)
# train XGBoost model
xgboost2 <- train(purchase ~ hhinc + hhsize + race + ethnicity +
                affluence + pres_child + age_head + educ_fem_head +
                census,data=data, tuneGrid = grid_default,
                method = "xgbTree", metric ="Accuracy")

[09:18:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:18:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:18:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:18:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:18:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:18:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:18:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:18:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:18:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:18:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:18:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:18:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:18:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:18:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:18:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:18:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:18:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:18:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:18:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:18:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:18:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:18:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:18:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:18:53] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:18:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:18:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:18:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:18:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:18:57] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:18:58] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:18:59] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:00] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:00] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:01] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:02] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:03] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:04] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:05] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:06] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:07] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:08] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:08] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:09] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:10] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:11] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:12] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:13] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:13] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:14] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:15] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:16] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:17] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:18] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:19] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:20] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:21] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:21] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:22] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:23] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:24] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:25] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:25] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:26] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:27] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:53] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:57] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:58] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:19:59] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:00] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:01] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:02] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:03] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:04] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:05] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:06] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:07] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:07] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:08] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:09] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:10] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:11] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:12] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:12] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:14] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:15] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:16] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:17] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:18] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:18] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:19] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:20] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:21] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:22] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:23] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:23] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:24] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:25] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:26] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:27] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:53] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:53] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:57] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:58] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:58] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:20:59] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:00] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:01] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:01] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:02] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:03] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:03] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:04] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:05] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:06] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:06] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:07] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:08] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:09] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:09] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:10] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:11] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:12] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:13] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:13] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:14] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:15] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:16] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:17] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:17] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.

plot(xgboost2)

xgboost2$bestTune

  nrounds max_depth  eta gamma colsample_bytree min_child_weight subsample
9     200         5 0.01     0              0.7               10       0.6

xgb.plot.tree(model = xgboost2$finalModel,
trees = 1, plot_width = 800, plot_height = 800)

importance_matrix <- xgb.importance(model = xgboost2$finalModel)
print(importance_matrix)

                             Feature        Gain       Cover  Frequency
                              <char>       <num>       <num>      <num>
 1:                    pres_childYes 0.313255945 0.149982238 0.07957560
 2:                    age_head< 35Y 0.213512031 0.140750397 0.08885942
 3:                     age_head65+Y 0.071533380 0.073617366 0.06100796
 4:                  hhsize4 or more 0.056827456 0.057961415 0.05238727
 5:                   age_head50-54Y 0.034538342 0.046493002 0.04045093
 6: educ_fem_headSome or High School 0.031720721 0.056185679 0.06432361
 7:                    raceCaucasian 0.031277181 0.048490198 0.06366048
 8:                   age_head35-44Y 0.028896302 0.041576967 0.05238727
 9:                   age_head45-49Y 0.027824912 0.034404236 0.02188329
10:                      censusSouth 0.024539377 0.038736697 0.07957560
11:                 hhsize3  members 0.021220160 0.033321589 0.03580902
12:                     hhinc$70-99K 0.021092874 0.036584409 0.04310345
13:                       censusEast 0.019982536 0.031279268 0.03183024
14:   educ_fem_headGradauted College 0.016224577 0.036359556 0.06432361
15:  educ_fem_headPost College Grade 0.016079962 0.030798136 0.03050398
16:                     hhinc$50-69K 0.015814372 0.028411720 0.03116711
17:      affluenceLiving Comfortably 0.014619607 0.027782896 0.04376658
18:                    censusCentral 0.014351058 0.029615329 0.04177719
19:                     hhinc$25-49K 0.013456453 0.026161236 0.03912467
20:              affluenceDoing Well 0.008700676 0.019426143 0.02453581
21:                ethnicityHispanic 0.002826491 0.006637517 0.00397878
22:                       hhinc< 25K 0.001705586 0.005424006 0.00596817
                             Feature        Gain       Cover  Frequency

xgb.plot.importance(importance_matrix = importance_matrix)

10 Extreme Gradient Boosting: nhefs data

nhefs <- read_csv(here("data", "nhefs.csv")) %>%
mutate(wt_delta = as.numeric(wt82_71 > median(wt82_71)))


# create tuning grid
grid_default <- expand.grid(nrounds = c(200,300),
max_depth = c(4,5), eta = c(0.01), gamma = c(0,1), min_child_weight = c(10, 25),
colsample_bytree = c(0.7), subsample = c(0.6))

set.seed(123)
# train XGBoost model
xgboost1 <- train(factor(wt_delta) ~ qsmk +
sex + age + income + sbp + dbp + price71 +
tax71 + race, data = nhefs, tuneGrid = grid_default,
method = "xgbTree", metric = "Accuracy")

[09:21:20] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:21] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:22] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:22] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:23] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:24] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:25] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:26] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:27] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:53] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:57] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:58] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:21:59] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:00] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:01] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:01] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:02] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:03] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:04] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:05] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:06] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:07] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:08] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:09] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:10] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:10] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:11] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:12] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:13] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:14] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:15] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:16] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:17] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:18] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:18] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:19] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:20] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:21] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:23] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:24] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:25] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:25] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:26] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:27] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:53] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:53] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:57] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:57] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:58] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:22:59] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:00] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:01] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:01] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:02] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:03] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:04] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:05] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:06] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:06] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:07] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:08] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:09] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:10] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:11] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:11] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:12] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:13] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:14] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:14] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:15] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:16] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:17] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:18] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:19] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:20] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:20] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:21] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:22] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:23] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:24] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:25] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:25] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:26] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:27] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:53] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:57] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:57] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:58] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:23:59] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:24:00] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:24:00] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:24:01] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:24:02] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:24:03] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:24:04] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:24:05] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:24:05] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:24:06] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:24:07] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:24:08] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:24:09] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:24:10] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[09:24:10] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.

# plot(xgboost1)
xgboost1$bestTune

   nrounds max_depth  eta gamma colsample_bytree min_child_weight subsample
15     200         5 0.01     1              0.7               25       0.6

xgb.plot.tree(model = xgboost1$finalModel,
trees = 1:2, plot_width = 800, plot_height = 800)

importance_matrix <- xgb.importance(model = xgboost1$finalModel)
print(importance_matrix)

   Feature        Gain      Cover   Frequency
    <char>       <num>      <num>       <num>
1:     age 0.458471328 0.32119934 0.281146637
2:     dbp 0.153948154 0.16985337 0.171995590
3:    qsmk 0.140593604 0.14179957 0.108048512
4:     sbp 0.096442882 0.13536373 0.153252481
5: price71 0.061374316 0.09701480 0.127894157
6:  income 0.046092537 0.06665409 0.072767365
7:     sex 0.022381198 0.03164424 0.044101433
8:   tax71 0.015433740 0.02554422 0.033076075
9:    race 0.005262242 0.01092664 0.007717751

xgb.plot.importance(importance_matrix = importance_matrix)